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Abstract. We give a recursive formula for the Mobius function of an interval 
[ct, tt] in the poset of permutations ordered by pattern containment in the case 
where tt is a decomposable permutation, that is, consists of two blocks where 
the first one contains all the letters 1,2, ... ,k for some k. This leads to many 
special cases of more explicit formulas. It also gives rise to a computationally 
efficient formula for the Mobius function in the case where a and tt are separa- 
ble permutations. A permutation is separable if it can be generated from the 
permutation 1 by successive sums and skew sums or, equivalently, if it avoids 
the patterns 2413 and 3142. A consequence of the formula is that the Mobius 
function of such an interval [a, tt] is bounded by the number of occurrences of 
(T as a pattern in tt. We also show that for any separable permutation tt the 
Mobius function of (1, tt) is either 0, 1 or —1. 



1. Introduction 

Let Sn be the set of permutations of the integers {1, 2, . . . , n}. The union of all 
Sn for n = 1,2,... forms a poset V with respect to pattern containment. That is, 
we define ct < tt in P if there is a subsequence of tt whose letters are in the same 
order of size as the letters in a. For example, 132 < 24153, because 2,5,3 appear in 
the same order of size as the letters in 132. We denote the number of occurrences 
of (7 in TT by cr(7r), for example 132(24153) = 3, since 243, 253 and 153 are all the 
occurrences of the pattern 132 in 24153. 

A classical question to ask for any combinatorially defined poset is what its Mobius 
function is. For our poset V this seems to have first been mentioned explicitly 
by Wilf [8]. The first result in this direction was given by Sagan and Vatter [5], 
who showed that an interval [a, tt] of layered permutations is isomorphic to a certain 
poset of compositions of an integer, and they gave a formula for the Mobius function 
in this case. A permutation is layered if it is the concatenation of decreasing 
sequences, such that the letters in each sequence are smaller than all letters in 
subsequent sequences. Further results were given by Steingrimsson and Tenner [7], 
who showed that the Mobius function /i(cr, tt) is whenever the complement of the 
occurrences of u in tt contains an interval block, that is, when tt has a segment of 
two or more consecutive letters that form a segment of values, where none of these 
consecutive letters belongs to any occurrence of a in tt. One example of such a pair 
is (132,598342617), where the letters 342 do not belong to any occurrence of 132 
in 598342617. Steingrimsson and Tenner fT' also described certain intervals where 
the Mobius function is either 1 or —1. 
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In this paper, we focus on permutations that can be expressed as direct sums or 
skew sums of smaher permutations. A direct sum of two permutations a and /?, 
denoted by a + /3, is the concatenation a/3', where /3' is obtained by incrementing 
each element of /? by \a\. For example, 31426587 can be written as a direct sum 
3142+2143. Similarly, a skew sum a * /3 is the concatenation a'/3 where a' is 
obtained by incrementing a by \/3\. 

A permutation that can be written as a direct sum of two nonempty permutations 
is decomposable. The decomposition of a permutation n is an expression tt = tti + 
7r2 + ■ • • + TTfc in which each sumniand tt^ is indecomposable. A permutation is 
separable if it can be obtained from the singleton permutation 1 by iterating direct 
sums and skew sums (for an alternative definition see Section [5]). 

Our main result is a set of recurrences for computing the Mobius function fi{a^ tt) 
when TT is decomposable. If tti + ■ • ■ + tt/j is the decomposition of tt, then these 
recurrences express ii{(t, tt) in terms of Mobius functions involving the summands tt^. 

In the special case when tt is separable, these recurrences provide a polynomial-time 
algorithm to compute fi{a, tt). These recurrences also allow us to obtain an alterna- 
tive combinatorial interpretation of the Mobius function of separable permutations, 
based on the concept of 'normal embeddings'. This interpretation of fi generalizes 
previous results of Sagan and Vatter [5] for layered permutations. 

Using these expressions of the Mobius function in terms of normal embeddings, 
we derive several bounds on the values of fi{a,Tr) for a and tt separable. In [7], 
Steingrfmsson and Tenner conjectured that for permutations a and tt avoiding the 
pattern 132 (or any one of the patterns 213, 231, 312) the absolute value of the 
Mobius function of the interval [a, tt] is bounded by the number of occurrences of a 
in TT. We prove this conjecture for the more general class of separable permutations 
(for arbitrary cr and tt this bound does not hold in general). In particular, if 
TT has a single occurrence of a then /i((T, tt) is either 1, or —1. We also prove a 
generalization of another conjecture mentioned in [7] , showing that for any separable 
permutation tt, /i(l,7r) is either 1, or —1. 

For a non-separable decomposable permutation tt, our recurrences are not sufficient 
to compute the value of ^(cr, tt). Nevertheless, they allow us to give short simple 
formulas in many special cases. 

For instance, suppose that a is indecomposable and that tt is decomposable and of 
length at least 3. Then we show that ii{a, tt) can only be nonzero if all the blocks in 
the decomposition of tt are equal to the same permutation tt' > 1, except possibly 
the first and the last block, which may be equal to 1. In such cases, fi{a,TT) equals 
(— 1) V(o', tt'), where i G {0, 1, 2} is the number of blocks of tt that are equal to 1. 

As another simple example, our results imply that when a and tt are permuta- 
tions with decompositions a = ai + cf2 and tt ~ tti + tt2, with tti and tt2 both 
different from 1, then fi{a,TT) = fi{ai,TTi)fi{a2,TT2) if tti ^ tt2, and fi{a,TT) = 

IJ.{ai,TTl)lJ,{a2,TT2) + A*(0-, TTl) if TTl=TT2. 

The paper is organized as follows: In the next section we provide necessary defini- 
tions. In Section 13] we present the main results, the recursive formulas for reducing 
the computation of the Mobius function of decomposable permutations to that of 
indecomposable permutations. Section |4] deals with the case of separable permu- 
tations and their normal embeddings. Finally, in Section [5] we mention some open 
problems, in particular questions about the topology of the order complexes of 
intervals in our poset, which we have not dealt with in the present paper. 
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2. Definitions and Preliminaries 

An interval [(J,tt] in a poset {V,<) is the set {p: a < p < tt}. In this paper, 
we deal exclusively with intervals of the poset of permutations ordered by pattern 
containment. 

The Mobius function p{a,TT) of an interval [cr, tt] is uniquely defined by setting 
fj,{a, a) = 1 for all a and requiring that 

pG[cr,7r] 

for every cr < vr. When ct ^ vr, we define p{a, tt) to be zero. 

An equivalent definition is given by Philip Hall's Theorem ^ Proposition 3.8.5], 
which says that 

(2) pi<J,7r)= E (-1)^^^)=E(-1)'^- 

C6£((T,7r) t 

where £(cr, tt) is the set of chains in [a, tt] that contain both a and tt, L{C) denotes 
the length of the chain C, and Ci is the number of such chains of length i in 
[cr, tt] . A chain of length z in a poset is a set of i + 1 pairwise comparable elements 
xq < xi < ■ ■ ■ < Xi. For details and further information, see [B]. 

The direct sum, o; + /3, of two nonempty permutations a and /3 is the permutation 
obtained by concatenating a and /?', where /?' is /? with all letters incremented by 
the number of letters in a. A permutation that can be written as a direct sum of two 
non-empty permutations is decomposable, otherwise it is indecomposable. Examples 
are 2314576 = 231 + 12 + 21, and 231, which is indecomposable. In the skew sum 
of a and /3, denoted by a * /?, we increment the letters of a by the length of /? to 
obtain a' and then concatenate a' and /3. For example, 6743512 = 12 * 213 * 12. 
We say that a permutation is skew-indecomposable if it cannot be written as a skew 
sum of smaller permutations. 

A decomposition of tt is an expression tt = tti + 7r2 + ■ • ■ + tt^ in which each summand 
TT, is indecomposable. The summands 7ri,...,7rfc will be called the blocks of tt. 
Every permutation tt has a unique decomposition (including an indecomposable 
permutation tt, whose decomposition has a single block tt). 

A permutation is separable if it can be generated from the permutation 1 by iterated 
sums and skew sums. In other words, a permutation is separable if and only if it is 
equal to 1 or it can be expressed as a sum or skew sum of separable permutations. 

Being separable is equivalent to avoiding the patterns 2413 and 3142, that is, con- 
taining no occurrences of either. Separable permutations have nice algorithmic 
properties. For instance, Bose, Buss and Lubiw [2] have shown that it can be de- 
cided in polynomial time whether a < tt when cr and tt are separable, while for 
general permutations the problem is NP-hard. 

It is sometimes convenient to allow permutations to have zero length, while in other 
situations, the permutations are assumed to be nonempty. The unique permutation 
of length 1 is denoted by 1, and the unique permutation of length is denoted 
by 0. We make it a convention that the permutation is neither decomposable 
nor indecomposable. In other words, whenever we say that a permutation tt is 
decomposable (or indecomposable), we automatically assume that tt is nonempty. 

Suppose that tt is a nonempty permutation with decomposition tti + • • ■ + 7r„ . For 
an integer i G {0, . . . ,n}, we let 7r<i denote the sum tti + tt2 + ■ ■ ■ + TTi and let 7r>i 
denote the sum tt^+i + • • ■ + 7r„. An empty sum of permutations is assumed to be 
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equal to 0, so in particular 7r<o = 7r>„ — 0. Any permutation of the form 7r<i for 
some i will be called a prefix of tt, and any permutation of the form 7r>j is a suffix 
of TT. Note that /i(0, 0) = 1, /i(0, 1) = —1, and it is easily seen that /i(0, r) = for 
any r > 1. 

3. The Main Results 

Throughout this section, we assume that cr is a nonempty permutation with decom- 
position ai + ■ ■ ■ + am and that tt = tti + • • • + 7r„ is a decomposable permutation 
(so n > 2 and, in particular, tt is nonempty). The goal in this section is to prove 
a set of recurrences that allow us to express the Mobius function /i((T, tt) in terms 
of the values of the form /i((T', tt'), where tt' G {tti, . . . , 7r„} is a block of tt and a' 
is a sum of consecutive blocks of a. Note that a may itself be indecomposable, in 
which case m is equal to 1 and ai = a. 

There are two main recurrences to prove, dealing respectively with the cases tti = 1 
and TTi > 1. 

Proposition 1 (First Recurrence). Let a and vr be nonempty permutations with 
decompositions a = ai + ■ ■ ■ + f7,„ and tt = tti + • ■ • + 7r„, where n > 2. Suppose 
that TTi = 1. Let k > I be the largest integer such that all the blocks tti, . . . ,7rfc are 
equal to 1, and let £ > be the largest integer such that all the blocks ai, . . . ,ai are 
equal to 1. Then 

r ifk-l>£, 
^J-i<7,^T) = < -fi{ayk-i,T^>k) ifk -1 = 1, 

[/i(cr>fe,7r>fc) - ^((T>fc_i,7r>fc) ifk-l<£. 

Note that the suffixes CT>fc_i, a^k and 7r>fe in the statement of Proposition [1] may 
be empty. This first recurrence shows how to compute the Mobius function when 
TT starts with 123 ... fc for some fc > 1. The second recurrence takes care of the 
remaining cases, that is, when tt does not start with 1. 

Proposition 2 (Second Recurrence). Let a and tt be nonempty permutations with 
decompositions a = ai + ■ ■ ■ + Um o,nd tt = tti + • • • + 7r„, where n > 2. Suppose 
that TTi > 1. Let k > 1 be the largest integer such that all the blocks tti, . . . , tt^ are 
equal to tti . Then 

m k 

(3) /i(cr, 71") = X! X! M(o-<i, 7ri)^(cr>,, 7r>j). 

Note that Propositions [T] and remain true when all the direct sums are replaced 
with skew sums, and the decompositions are replaced with skew decompositions. 
To see this, it is enough to observe that if tt denotes the reversal of tt (i.e., tt is 
the permutation obtained by reversing the order of elements of tt), then fi(cr,TT) = 
IJ,{a, tt) for any a and tt, since [a, tt] and [a, n] are isomorphic posets. 

Before we prove the above two recurrences, we give three corollaries to provide some 
idea of how the second recurrence can be used. When we write k x tt we mean a 
sum TT + TT + ■ ■ ■ + TT with k sumniands. 

Corollary 3. Let a, tt and k be as in Proposition\^ and suppose that a is inde- 
composable, that is, m = I. Then 

{^(fj, TTl), ifTT = kXTTl 

— /i(cr, TTl), ifTT=kXTTl+l 
0, otherwise 
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Proof. Since m = 1, Equation ([3]) takes the form 



k 

= ^^(o-<i,7ri)^((7>i,7r>j) 

k 

^^^^{<^, 7ri)Ai(0,7r>j) 
= /i(cr, 7ri)^(0,7r>fe), 



where the last equahty follows from the fact that /x(0,7r>j) is equal to whenever 
7r>j has more than one block. 

We have /i(0, 7r>fe) — 1 when 7r>fc = 0, /i(0,7r>fc) = —1 when 7r>fc = 1, and 
/i(0, 7r>i;) = otherwise. In particular, /i(0, 7r>i:) can only be nonzero either when 
k ~ n and n = k x tti, or when k ^ n — 1 and 7r = fcx7ri + l. □ 

Corollary [3] implies that if a is indecomposable and tt is decomposable, then almost 
always ^{(t,tt) — 0, since the two exceptions for tt given in the corollary are of 
a proportion that clearly goes to zero among decomposable permutations as their 
length goes to infinity. 

Corollary 4. With a and tt as in Proposition assume that a and tt decompose 
into exactly two blocks, with a = a\ ^ and n — tti + tt2, and that 7ri,7r2 > 1. 
Then 

fi{(Jl,Tri)^l{cr2,1T2), if TriT^TT2 

^(f7l,7ri)/x(fT2,7r2) +/^(cr,7ri) if m = TT2 



//(cr,7r) 



Proof. If TTi ^ 772 (so fc = 1), then the summation in Equation expands into 

/i(cr,7r) = /i((Ti,7ri)^((T2,7r2) + /i((t, 7ri)^(0, 7r2). 

Since tt2 > 1, the second summand vanishes and /i((T, tt) ~ (u(cti, 7ri)/i(CT2, 7r2). 

If, on the other hand, tti = 7r2, then Equation (jSj states that ii{a,TT) is equal to 

^(cri,7ri)^((T2,7r2) + ^^(o-l, 7ri)M((T2, 0) + ^(cr, 7ri)^(0, 712) + /.t(cr, 7ri)/i(0, 0) 

=^((Ti,7ri)^(o-2,7r2) + n 

Remark 5. An obvious question to ask is whether the product formula ii{a, n) = 
/x(cri,7ri)/i((T2,7r2), in the case when tti ^ 7r2, is a result of the interval (tT,7r) being 
isomorphic to the direct product of the intervals [ci, tti] and [0-2, 772]. Although this 
seems to occur frequently, it does not hold in general. 

The following corollary is an immediate consequence of Proposition [T] (the case 
when k-l^e = 0). 

Corollary 6. Suppose a and tt are permutations of length at least two, such that 
neither begins with 1. Then fi{(7, 1 + tt) = — /i((T, tt). 

Both recurrences (Propositions [T] and [J) are proved using arguments that involve 
cancellation between certain types of chains in the poset of permutations. Let us 
first introduce some useful notation. For a chain C = {ao < ai < • • • < Offe} of 
permutations let L{C) denote the length of C, which is one less than the number 
of elements of C. The weight of C, denoted by w{C), is the quantity (— 1)^^'-^'. If 
£ is any set of chains, then the weight of £ is defined by 

«;(£)= 5:^0= 5: (-l)^(^). 
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Recall that £((t, tt) is the set of all the chains from cr to tt that contain both a 
and TT. We know that fi{(T,TT) ~ w(£((7, tt)), by Philip Hall's Theorem. 

For a chain C — {ao < ai < ■ ■ ■ < a^} and a permutation /3, we let /3 + C denote 
the chain {/3 + ao < l3 + ai < • ■ • < (3 + ak}- The chain C + /3 is defined analogously. 

3.1. Proof of the first recurrence. Let us now turn to the proof of Proposition[TJ 
Suppose that cr, tt, m, n, k, and £ are as in the statement of the proposition. For 
a permutation t € S, define the degree of t, denoted by deg(r), to be the largest 
integer d such that r can be expressed as d x 1 + t' for some (possibly empty) 
permutation r'. In particular, we have k = deg(7r) and i — deg(tT). 

Let C — {tq < Ti < • • • < Tp} be a chain of permutations. We say that a permu- 
tation Ti G C is the pivot of the chain C, if deg(Ti) < deg(Tj) for each j > j, and 
deg(Ti) < deg(Tj) for each j < i. In other words, the pivot is the element of the 
chain with minimum degree, and if there are more elements of minimum degree, 
the pivot is the largest of them. 

Let p denote the permutation 7r>i. Obviously deg(p) = fc — 1 and 1 + p — tt. We 
partition the set of chains £(iJ, tt) into three disjoint subsets, denoted by £q, 
and £c, and we compute the weight of each subset separately. A chain C G €{a, tt) 
belongs to £o if its pivot is the permutation tt, the chain C belongs to £{, if its pivot 
is the permutation p, and C belongs to £c otherwise. We now separate the main 
steps of the proof into independent claims. 

Claim 7. // deg(cr) < deg(7r) (so £ < k), then €a is empty. Otherwise, w(£a) = 

Proof. Obviously, if deg(cr) < deg(7r), then no chain from cr to tt can have tt for 
pivot, because the pivot must have minimal degree among the elements of the chain. 
Thus, £q is empty. 

Assume now that deg(cr) > deg(7r). We show that there is a length-preserving 
bijection between the set of chains £(cr>fe, 7r>fc) and the set of chains £„■ Indeed, 
take any chain C G C(cr>fe, 7r>fc), and create a new chain /(C) — {kxl) + C. Then 
/(C) is a chain from cr to tt, and since every element of /(C) has degree at least k, 
while TT has degree exactly k, we see that tt is the pivot of /(C). Hence /(C) G £a- 

On the other hand, if C is any chain from £q, we see that each element of C 
has degree at least k, because tt has degree k and is the pivot of C. Thus, every 
element r' G C is of the form fc x 1 + r for some r, and hence there exists a 
chain C G €{a^k,'^>k) such that C = /(C). Since / is clearly injective and 
length-preserving, we conclude that w(£a) = w(£(cr>fe, 7r>fc)) = fJ-{<^>k,'^>k), as 
claimed. □ 

Claim 8. //deg(cr) < deg{p) (so I < k — 1), then €b is empty. Otherwise, w{€b) = 
— M(cr>fe_i,7r>fc). 

Proof. If deg (cr) < deg(p) then p cannot be the pivot of any chain containing cr and 
£b is empty. 

Assume now that deg(cr) > deg(p). We will describe a parity-reversing bijection 
/ between the set of chains £(cr>fc_i, 7r>fc) and the set of chains Take a chain 
C G e:(cr>fe_i,7r>fe). Define a new chain C by C = {{k - 1) x 1) -|- C. Notice that 
C is a chain from a to p whose pivot is p and whose length is equal to the length 
of C. Define the chain /(C) by /(C) = C U {tt}. Then the chain /(C) belongs to 
£(, and has length L{C) + 1. It is again easy to see that / is a bijection between 
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£(cr>fe_i, 7r>i;) and which shows that 

w{^b) = -w(e:(CT>fe_i,7r>fe)) 



Ai(o'>fc-i, 7r>fe), 



as claimed. 



□ 



Claim 9. w{€c) = 0. 

Proof. We construct a parity-reversing involution /: ^c- Let C be a chain 

from £c, let r be its pivot, and let r' be the successor of r in C. By definition of £c, 
T is not equal to tt, so r' is well defined. From the definition of a pivot, we know 
that deg(r) < deg(T'). Let us distinguish two cases: 

(1) If r' = 1 + T, we define a new chain /(C) by /(C) = C\ {r'}. Note that in 
this case, we know that r' is different from tt, because otherwise r would 
be equal to p, contradicting the definition of £c- Thus, /(C) G £c- Note 
that r is a pivot of /(C). 

(2) If t' ^ 1 + T, then we easily deduce that t' > 1 + t (recall that deg(r') > 
deg(T)). We then define a new chain /(C) = C U {1 + r}, in which the new 
element 1 + r is inserted between t and r' . 

The mapping / is easily seen to be an involution on the set €c which preserves the 
pivot and maps odd-length chains to even-length chains and vice versa. This shows 
that w{€c) — 0, as claimed. □ 

From these claims, Proposition [T] easily follows. Indeed, Claim [S] implies that 
fi{a;7T) — w(£q) +w{€b)- From Claims [7] and [5] we deduce the values of fj,{a,7r): 

• li k — 1 > £ then both €a and £b are empty and /i((T, tt) = 0. 

• If A; — I = ^ then is empty and /x(cr, tt) = w{€b) = --/i(cr>fe_i, 7r>fe). 

• Uk-1 < e, then /x((7, tt) = w(£a) + w{€b) = At(o->fe, 7r>fe) - /z(cr>fe-i, 7r>fe). 

This completes the proof of Proposition [TJ 

3.2. Proof of the second recurrence. It remains to prove Proposition [21 The 
proof is again based on cancellation among the chains from a to tt. Before stating 
the proof, we need more terminology and several lemmas. 

Let a, j5 and p be any permutations. We say that a is a p-tight subpermutation of 
;9, denoted by a < /?, if a < /3 but p -I- a is not contained in /?. We say that a chain 
{ao < ai < • • • < Q!fe} is p-tight if ai^i < ai for every i — 1, . . . ,k. Let C(a, /3) be 
the set of all the p-tight chains from a to /3. 

The following simple properties of p-tightness follow directly from the definitions, 
and they are presented without proof. 

Lemma 10. For arbitrary permutations a, 13, 7 and p, we have a + 'j < /3 + j if 
and only if a < (3. 

Lemma 11. If p is a nonempty indecomposable permutation, and if a and 13 are 
arbitrary permutations, then p-t-Q;<p-f/3 if and only if a < (3. 

The next lemma shows the relevance of p-tightness for the computation of p. 

Lemma 12. Let (3 be a permutation with decomposition /3 — Pi + 132 + ■ ■ ■ + l3p. 
Let p be a nonempty indecomposable permutation, and let a be any permutation. 



(1) Ifp^(3x, thenp,{a,(3) 

(2) Ifp = l3i, thenti{a,l3) 



w{€P{a,l3)). 

w{(tP{a,l3))-w{(t''{a,l3>i)). 
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Proof. Let us first deal with the first claim of the lemma. Let us define £ = 
€{a, (3) \ C(a,/3) to be the set of all the chains from a to /? that are not p-tight. 
The first part of the lemma is equivalent to saying that = 0. To prove this, 

we find a parity-reversing involution / on the set £. 

Consider a chain C — {a — ao < ai < ■ ■ ■ < aq — 13} £ €. Since C is not p-tight, 
there is an index i such that p + ai < a^+i. Fix the smallest such value of i. We 
distinguish two cases: either p + ai < ai+i, or p + ai = ai+i. 

li p + ai < Qfi+i, define a new chain 

/(C) = CU {p + ai} = {a = ao < ai < ■ ■ ■ < ai < p + ai < ai+i < ■ ■ ■ < aq = /3}. 

On the other hand, if p + ai — cti+i, define a new chain 

/(C) = C \ {p + a,} = {a = ao < ai < ■ ■ ■ < a, < ai+2 < ■ • • < a, = /?}. 

Note that, since we assume that p ^ Pi and that p is indecomposable, we know 
that p + ai is not equal to /3. Moreover, in the chain /(C) the element ai is not 
a p-tight subpermutation of its successor in the chain. Thus, we see that /(C) is 
a chain from £. It is easy to see that / is an involution, and that it reverses the 
parity of the length of the chain, showing that w(£) = 0. This proves the first part 
of the lemma. 

Let us prove the second part. Assume that p — (3i, that is, /3 = p + /3>i. Consider 
a chain C from a to /?, and let ao, ai, . . . ,aq be the elements of C. We say that 
the chain C is almost p-tight if its second largest element aq-i is equal to /3>i and 
if Ui-i < ai for each i < q ~ 1. Note that an almost p-tight chain is never p-tight, 
because /3>i is not a p-tight subpermutation of /3 = p + /3>i. 

We partition the set £(a, f3) into three disjoint sets Cq, £b, and £c, where is the 
set C(a, (3) of p-tight chains, €b is the set of almost p-tight chains, and £c contains 
the chains that neither p-tight nor almost p-tight. 

Consider again the mapping / defined in the proof of the first part of the lemma. 
This mapping, restricted to the set £c, is easily seen to be a parity-reversing involu- 
tion on £ci which shows that w{€c) = 0. This means that p(a, (3) = u;(£a) + ^(£6)- 

Furthermore, note that an almost p-tight chain from a to /3 consists of a p- 
tight chain from a to /3>i followed by /3, and conversely, any p-tight chain from 
a to /3>i can be extended to an almost p-tight chain from a to /3 by adding 
the element /?. Thus, we see that w(£&) = — w(£''(a, /3>i)). This implies that 
p{a, (3) = w{ilP{a, (3)) - w{€P{a, /3>i)), as claimed. □ 

The next lemma is an easy consequence of Lemma 1121 

Lemma 13. Let (3 he a permutation with decomposition P = (3i + 132 + ■ • ■ + l3p. 
Let p be an indecomposable permutation, and let a be any permutation. Let q > 
be the largest integer such that the blocks /3i,/32, ■ ■ ■ , f3q are all equal to p. Then 

9 

7«(G:''(a,/3)) =^p(a, /?>,). 

i=0 

Proof. Proceed by induction on q. If g = 0, the claim reduces to the identity 
w{€.P{a, /3)) — ij,{a,(3), which follows from the first part of Lemma \T% Suppose 
that q > 0. Then the second part of Lemma [T2] applies and we get that 



p{a, 13) = w{(tPia, 13)) ~ «;(£"(«, /3>i)), 
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w{^P{a, 13)) = fi{a, 13) + u;(e:''(a, /3>i)). 




^(£"(0, /3>i)) = il3>l)>^) = Yl ^>»)- 




Combining this with we obtain the desired identity. 



□ 



Before we proceed towards the proof of Proposition [21 we need to introduce more 
definitions. Let /? be a permutation with decomposition /3i + ■ • • + /3p into indecom- 
posable blocks, let a be any permutation. Let C be a chain of permutations, with 
elements a = ao < ai < • • • < = /?. We express each element at of the chain 
as a sum of two permutations, called head and tail, denoted respectively as hi{C) 
and ti(C), with — hi{C) + ti{C). The head and tail are defined inductively as 
follows; for i = q, we have ai = aq = (3 and we define hq{C) — /3i and tq{C) = /3>i. 

Suppose now that the head and tail of have been already defined, and let us define 
head and tail of a^-i. Let us put 7 = a^-i, and assume that 7 has decomposition 
71 + 72 + • • • + 7r into indecomposable blocks. Let j be the smallest integer such 
that 7>j < tiiC). It then follows that 7<j < hi{C). We define hi^iiC) = 7<j and 
ti_i(C) = 7>j. In other words, the tail of ai_i is its longest suffix that is contained 
in the tail of Ui . 

If the chain C is clear from the context, we write hi and ti instead of hi{C) and 
ti{C). Note that ho < hi < ■ ■ ■ < hq and to < ti < ■ ■ ■ < tq. 

We say that the chain C of length q is split if there is an index s £ {0, . . . ,q} 
such that to = ti = ■ ■ ■ — ts and hs — hs+i — ■ ■ ■ — hq. Such an index s is then 
necessarily unique. The next lemma demonstrates the relevance of these notions. 

Lemma 14. Let j3 be a permutation with decomposition /3i + /32 + • ■ • + /3p such 
that /3i ^ 1. Let a be an arbitrary permutation. Let €.* he the set of all the chains 
from €,{a,l3) which are split and 1-tight. Then ii{a,l3) = w{€-*). 

Proof. By the first part of Lemma [T^ we know that ^{a, f3) is equal to w{(i^ {a, (3)), 
that is, to the total weight of all the 1-tight chains from a to (3. Define the set 
£ = £"""(0, /3) \ £* of all the 1-tight, non-split chains from a to /?. 

To prove the lemma, we need to show that w{€) — 0. To achieve this, we again use 
a parity-reversing involution / on the set £. Consider a chain C G £ with elements 
ao < ai < ■ ■ ■ < aq. Since C is not split, there must exist an index j G {1, . . . , g} 
such that either 

(1) hj^i < hj and tj^i < tj, or 

(2) hj^i — hj < /ij+i and tj^i < tj — ij+i. 

Fix such an index j as large as possible and distinguish two cases depending on 
which of the two above-mentioned possibilities occur for this index j. 

Case (HI). Assume that hj^i < hj and tj^i < tj. Let us write h — /ij-i, H ~ hj, 
t = ij-i, and T = tj, so we have a^-i = h+t and aj = H+T. Define a permutation 
^ — h + T, and a new chain /(C) = C U {7}. Note that since C is a 1-tight chain, 

and in particular a^-i < aj, we also have a^-i < 7 < a^-, and hence /(C) is a 1-tight 
chain as well. 

We need to prove that /(C) G £, which follows easily from the following claim. 
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Claim 15. Each permutation of C has the same head and tail in f{C) as in C . 
The permutation "f = h + T has head h and tail T in f{C'). 

Proof of Claim [75l It is clear tliat ttie claim holds for all the permutations that are 
greater than 7. 

It is also easy to see that the claim holds for 7. Indeed, the successor of 7 in /(C) 
is the permutation H + T, whose tail is T. Since the tail of 7 cannot be greater 
than T and since 7 = + T, it follows that the tail of 7 is T and its head is h. 

Let us now consider the permutation aj-i = h + 1. The successor of a^-i in C is 
the permutation aj = H + T, and the successor of aj-i in /(C) is the permutation 
^ = h + T . Since the two successors have the same tail T, and since the tail of 
a permutation only depends on the tail of its successor, we see that Oij-i has the 
same tail (and hence also the same head) in /(C) as in C. 

From these facts, the claim immediately follows. □ 

We may now conclude that /(C) G £, and turn to the second case of the proof of 
the lemma. 

Case Assume now that hj^i — hj < /ij+i and tj^i < tj = i^+i. Let us define 
h = /ij-i — hj, H — /ij+i, t = tj-i, and T — tj = ij+i. In particular, aj-i = h + t, 
a.j =h + T, and a.j+i = H + T. Define the chain /(C) = C \ {aj}. 

We claim that /(C) is 1-tight. To see this, it is enough to prove h + 1< H + T. 
Assume, for a contradiction, that 1 + h + t < H + T. In any occurrence of 1 + h + t 
inside H + T, the prefix 1 + h must occur inside H, otherwise we get a contradiction 
with the assumption that t is the tail of aj-i. This shows that 1 + h < H , and 
hence l + h + T= l + aj < aj+i = H + T, contradicting the assumption that C 
is 1-tight. To finish the proof of the lemma, we need one more claim. 

Claim 16. Each permutation of f{C) has the same head and tail in f{C) as in C. 

Proof of Claim [751 It is enough to prove the claim for the permutation a^-i = h+t, 
because any other permutation of /(C) has the same successor in /(C) as in C. For 
ctj-i, the claim follows from the fact that the successor of aj-i in C has the same 
tail as the successor of a^-i in f{C). This completes the proof of the claim. □ 

We now see that even in this second case, /(C) belongs to £. 

Combining the two cases described above, we see that / is a parity-reversing invo- 
lution of the set £. This means that w(£) = 0, and consequently, iJL{a, (3) — w{C*), 
as claimed. This completes the proof of the lemma. □ 

Finally, we can prove Proposition [51 Assume that cr is a permutation with decom- 
position ai + - ■ ■ + Um and that tt is a permutation with decomposition tti -I- • ■ • -I- 7r„ , 
where n > 2 and tti > 1. Let k > 1 he the largest integer such that all the blocks 
TTi , . . . , TTfe are equal to tti . Recall that our goal is to prove identity ([3]) , which reads 
as follows: 



Let £* be the set of 1-tight split chains from a to tt. From Lemma [HI we know 
that ^J.{a■,^^) = w{€*). For a chain C € €* , let to(C) be the tail of the element 
a £ C, which is the smallest element in the chain. By definition, to(C) is a suffix 



m 



k 
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of cr, that is, it is equal to cr>i for some value of i e {0, . . . ,m}. Define, for each 
I e {0, . . . , to}, the set of chains 

e, = {cer,<o(c) = (T>j. 

The sets <Zi form a disjoint partition of £*. We will now compute the weight of the 
individual sets 

Claim 17. Let C be a chain from €* . Every element of C has nonempty head. 
Consequently, to(C) is never equal to a, and hence £o is empty. 

Proof. Suppose that C has an element with empty head. Let a be the largest such 
element. By definition, the element tt e C has head equal to tti, so a 7^ tt. In 
particular, a has a successor a' in C, and a' has nonempty head. Let h' and t' be 
the head and tail of a'. By assumption, h' is nonempty, which means that 1 < h'. 
Moreover, a < t' , because a is its own tail. This means that 1 + a < a', which is 
impossible because the chains in £* are assumed to be 1-tight. 

This shows that every element of C has nonempty head, and the rest of the claim 
follows directly. □ 

Claim [TTI implies that w(£o) — 0; and hence /i(cr, tt) = X]i>i "^(^i)- K remains to 
determine the value of w{iLi) for i > 0. 

Fix an integer i G {1, . . . ,to}. Define h = (T<j, t = (T>i, H = tti, and T — 7r>i. 
Note that in a chain C ^ the permutation a has head h and tail t, while the 
permutation tt has head H and tail T. 

Claim 18. With the notation as above, 

w{€^) = w{€^{h,H))w{€"{t,T)). 

Proof. Let us write £' = €^{h,H) and £" = €"{t,T). We wiU provide a bijection 
/: C X £" — T' £i, which maps a pair of chains (Ci,C2) G £' x £" to a chain 
/(Ci,C2) G £i whose length is equal to L(Ci)+L(C2). Such a bijection immediately 
implies the identity — w{€')w{€") from the claim. 

The definition of the mapping / is simple: for Ci G €.' and C2 G define /(Ci, C2) 
to be the concatenation of the two chains Ci +t and + C2. This is well defined, 
since the maximum of Ci + i is the permutation H + t, which is also equal to the 
minimum of the chain H + C2. Thus, /(Ci,C2) is a chain of length L{Ci) + L{C2). 
Let us denote this chain by C. 

We now show that C belongs to C^. Let us call the two sub-chains Ci+t and H + C2 
respectively the bottom part and the top part of C. Note that the permutation H + t 
is the unique element of C belonging both to the top part and to the bottom part. 

By construction, C is a chain from a to tt. The bottom part of C is a 1-tight chain, 
because Ci was assumed to be 1-tight (see Lemma [TU)) . Similarly, by Lemma [Til 
the top part of C is a 1-tight chain, because C2 is iJ-tight and H is indecomposable. 
This shows that the chain C is 1-tight. 

Our next step is to prove that every element in the top part of C has head equal 
to H, and that every element in the bottom part of C has tail equal to t. Assume 
that this statement is false, and let a be the largest element of C for which it fails. 
Clearly, a ^ n, so a has a successor /3 in C. Suppose first that a belongs to the 
top part of C. Then a can be written as a sum H + a' for some a' G C2, and 
likewise /3 = iJ -I- /?' for /3' G C2 . By the choice of a, we know that the head of /3 
is H and hence its tail is /?'. Since a' < /3' , the tail of a contains a' . On the other 
hand, the only suffix of a longer than a' is the permutation a itself, because H is 
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indecomposable. By Claim [T71 the head of a must be nonempty, which means that 
the head of a can only be equal to H, which contradicts our choice of a. 

Suppose now that a does not belong to the top part of C. Then (3 belongs to the 
bottom part of C (and possibly to the top part as well). Consequently, a can be 
written a.s a' + 1 and (3 can be written as P' + t, with a',/3' G Ci. We also know 
that t is the tail of /3. This makes it clear that t is the tail of a as well, which is a 
contradiction. 

This proves that all the elements of the top part of C indeed have head H, and all 
the elements in the bottom part have tail t. This shows that C is a split chain and 
also that to{C) — t. We have shown that C E Ci. 

It is clear that / is an injective mapping. To complete the proof of the claim, it 
only remains to show that / is surjective, that is, for every C G there are chains 
(Ci, C2) e £' X £" with /(Ci, C2) = C. 

Choose a chain C E €i. Since C is split, it must contain the element H + t. Call the 
elements of C contained in H + t the bottom part of C, and the elements containing 
H + t the top part of C. The definition of split chain further implies that all the 
elements in the top part have the same head H and all the elements in the bottom 
part have the same tail t. Hence, the bottom part of the chain C has the form 
Ci + t for some chain Ci G H). Similarly, the top part has the form H + C2 
for a chain C2 G €.{t,T). Since C is 1-tight, we may use Lemmas [TUl and [TTI to 
see that Ci is 1-tight and C2 is iJ-tight, showing that (Ci,C2) £ £' x €" . Since 
/(Ci, C2) = C, we see that / is the required bijection. □ 

We now have all the necessary ingredients to finish the proof of Proposition [2l Let 
us write H — ni and T = 7r>i. From our results, we get 

^(cr,7r) = w(£(CT,7r)) 
= «;(£*) 

m 
m 

= E w{<t\a<,, H))w{€"{a>,, T)) 

m 

m k~l 

m k 
i=l 3 = 1 

Thus, Proposition [2] is now proved. 
We now present some consequences of Propositions [T] and [5] 

Corollary 19. There is an algorithm that, given two separable permutations a and 
TT, computes the value o//i((J, tt) in time polynomial in \a\ + \tt\. 

Proof. Let tt = 7ri7r2 • • • 7r„ be a separable permutation. For two integers i,j with 
^ ^ i l£ j ^ n, let 7r[i,j] denote the subpermutation of vr order-isomorphic to the 



by Lemma [TJ] 
by Claim HZ] 

by Claim HI] 

by first part of Lemma [12] 
by Lemma 1131 
since T>j = 7r>j+i 
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sequence 7ri,7ri-|_i, . . . ,7rj. Note that 7r[i,j] is also separable. We call 7r[i, j] a range 
subpermutation of n. 

Suppose that a = ai ■ ■ ■ am and tt = tti • • • 7r„ are two separable permutations. Our 
goal is to compute /i(CT, tt). We will use a straightforward dynamic programming 
algorithm to perform this computation. We will compute all the values of the form 
fi{(7[i, j],TT[k, £]), for all quadruples k,i) satisfying I < i < j < m and 1 < fc < 
£ < n. For each such quadruple {i, j, fc, i) we store the value of /i(cr[i, j] , 7r[fc, £]) once 
we compute it, so that we do not need to compute this value more than once, even 
though we may need it several times to compute other values of fi. 

There are O(m^n^) quadruples k,£) to consider, and for each such quadruple, 
we may use Propositions [T] and [2] to express ^{(j[i,j],Tr[k,£]) as a combination of 
polynomially many values of the form ^{a[i' ,j'],n[k' ,£']) where a[i',j'] and 7r[fc', £'] 
are range subpermutations of and 7r[/c,£] with 7r[fc',f'] ^ n[k,£]. Therefore, 

we can in polynomial time compute all the values of the form ^((T[i, j], 7r[fc, 
including /i(cr, tt) = /x(cr[l, m], 7r[l, n]). □ 

Note that the number of permutations belonging to an interval [a, tt] may in general 
be exponential in the size of tt, even when tt and a are separable. Therefore, 
computing the Mobius function /x(a', tt) directly from equation ([T]) would be much 
less efficient than the algorithm of the previous corollary. 

Let us say that a class of permutations C is sum- closed if for each 7T,a E C, the 
class C also contains tt + a. Similarly, C is skew-closed if tt, cr G C implies tt * a E C. 
For a set V of permutations, the closure of V, denoted by cl(7^), is the smallest 
sum-closed and skew-closed class of permutations that contains V. Notice that 
cl({l}) is exactly the set of separable permutations. 

The next corollary is an immediate consequence of Propositions [T] and [2] (see also 
Corollary [3]) , and we omit its proof. 

Corollary 20. Suppose that a is a permutation that is neither decomposable nor 
skew-decomposable. Let V be any set of permutations. Then 

max{ I ^ (cr, tt) I; tt G P} max{|/i(cr, 7r)|; tt G cl{P)}. 

Moreover, the computation of /i((T, tt) for tt G cl{'P) can be efficiently reduced to the 
computation of the values ii{a, p) for p E V. 

4. The Mobius function of separable permutations 

Let us now consider the values of fj,{cr, tt) for separable permutations a and tt. Our 
goal is to show that the values of the Mobius function in the poset of separable 
permutations have a combinatorial interpretation in terms of the so-called normal 
embeddings, which we define below. This alternative interpretation of the Mobius 
function generalizes previous results of Sagan and Vatter [5] for the Mobius function 
of intervals of layered permutations, which we explain at the end of this section. 

As a consequence of this new interpretation of the Mobius function, we are able 
to relate the Mobius function p{a,TT) to the number of occurrences of a in tt, by 
showing that |^((t, 7r)| < cr(7r). We also show that /x(l,7r) is equal to —1, or 1 
whenever tt is separable. 

The recursive structure of separable permutations makes it convenient to represent 
a separable permutation by a tree that describes how the permutation may be 
obtained from smaller permutations by sums and skew sums. We now formalize 
this concept. A separating tree T is a rooted tree T with the following properties: 
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• Each internal node of T has one of two types: it is either a direct node or 
a skew node. 

• Each internal node has at least two children. The children of a given internal 
node are ordered into a sequence from left to right. 

Each separating tree T represents a unique separable permutation tt, defined recur- 
sively as follows: 

• If T has a single node, it represents the singleton permutation 1. 

• Assume T has more than one node. Let A^i, . . . , Nk be the children of the 
root in their left-to-right order, and let Ti denote the subtree of T rooted at 
the node Ni. Let pi, . . . ,pfc be the permutations represented by the trees 
Ti, . . . , Tfe. Then T represents the permutation pi + - ■ ■ +pk if the root of T 
is a direct node and pi * ■ ■ ■ * Pk if the root of T is a skew node. 

Note that the leaves of T correspond bijectively to the letters of tt. In fact, when 
we perform a depth-first left-to-right traversal of T, we encounter the leaves in the 
order that corresponds to the left-to-right order of the letters of tt. See Figure [1] 
for an example. 

A given separable permutation may be represented by more than one separating 
tree. A separating tree is called a reduced tree if it has the property that the 
children of a direct node are leaves or skew nodes, and the children of a skew node 
are leaves or direct nodes. Each separable permutation tt is represented by a unique 
reduced tree, denoted by T(7r). We assume that each leaf of T is labelled by the 
corresponding letter of tt. 

This slightly modified concept of separating tree and its relationship with separable 
permutations have been previously studied in algorithmic contexts j2j [9] . We will 
now show that the reduced tree allows us to obtain a simple formula for the Mobius 
function of separable permutations. 

Let [n] denote the set {1, . . . , n}. Let tt = 7ri7r2 ■ ■ - Tin and a = (7if72 . . . CTm be two 
permutations, with cr < tt. An embedding of cr into tt is a function / : [m] — [n] 
with the following two properties: 

• for every i, j G [to], ii i < j then /(i) < /(j) (so / is monotone increasing). 

• for every i, j G [m], if Ui < aj, then T^f(i) < f(j) (so / is order-preserving). 

Let / be an embedding of a into tt. We say that a leaf £ of T{'k) is covered by the 
embedding / if the letter of tt corresponding to (. is in the image of /. A leaf is 
omitted by / if it is not covered by /. An internal node is a node that is not a leaf. 
An internal node N of T{tt) is omitted by / if all the leaves in the subtree rooted at 
N are omitted. A node is maximal omitted, if it is omitted but its parent in T(7r) 
is not omitted. 

Assume that tt is a separable permutation and T{-k) its reduced tree. Two nodes 
A'^i and N2 of a tree T(7r) are called twins if they are siblings having a common 
parent P, they appear consecutively in the sequence of children of P, and the two 
subtrees of T rooted at A^i and A''2 are isomorphic, that is, they only differ by the 
labeling of their leaves, but otherwise have the same structure. In particular, any 
two adjacent leaves are twins. 

A run under a node iV in T is a maximal sequence Ni , . . . , of children of TV 
such that each two consecutive elements of the sequence are twins. Note that the 
sequence of children of each internal node is uniquely partitioned into runs, each 
possibly consisting of a single node. A leaf run is a run whose nodes are leaves, and 
a non-leaf run is a run whose nodes are non-leaves. The first (leftmost) element 
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of each run is called the leader of the run and the remaining elements are called 
followers. 

Using the tree structure of T(t:), we will show that /i((T, tt) can be expressed as 
a signed sum over a set of embeddings of a into tt that have a special structure. 
Following the terminology of Sagan and Vatter ^ , we call these special embeddings 
normal. 

Definition 21. Let a and tt be separable permutations, let T(7r) be the reduced 
tree of tt. An embedding f of a into tt is called normal if it satisfies the following 
two conditions. 

• If a leaf £ is maximal omitted by /, then £ is the leader of its corresponding 
leaf run. 

• If an internal node N is maximal omitted by /, then is a follower in its 
non-leaf run. 

Let N{a,TT) denote the set of normal embeddings of a into tt. The defect of an 
embedding / G N{a, tt), denoted by d{f), is the number of leaves that are maximal 
omitted by /. The sign of /, denoted by sgn(/), is defined as (— l)'''^''^^. 

We now present our main result. 

Theorem 22. If a and tt are (possibly empty) separable permutations, then 

/^(c^>7r)= sgn{f). 

feN{a,7T) 

Consider, as an example, the two permutations tt and cr depicted on Figure [TJ The 
children of the root of T(tt) are partitioned into three runs, where the first run has 
three internal nodes, the second run has a single leaf, and the last run has a single 
internal node. Accordingly, there are five normal embeddings of a into tt, depicted 
in Figure [31 Of these five normal embeddings, two have sign -1 and three have sign 
1, giving /z(ct, tt) = 1. 




1 2 4 5 7 8 11 12 



Figure 1 . The separating trees of two permutations a and tt 

Proof of Theorem\^ Let 7i(cr, tt) denote the value of X]/e7V(cr tt) sgn(/). Our goal 
is to prove that /i(o', tt) is equal to /i((T, tt). We proceed by induction on |7r|. For 
a = TT, we clearly have 'p,{a,TT) — fi{a,TT) ~ 1, and if tt does not contain a, then 
7l(cr,7r) = fi{a,TT) = 0. 

Suppose now that a < tt. Since tt is separable, it is decomposable or skew- 
decomposable. Assume, without loss of generality, that tt is decomposable. Let 
TTi + ■ • ■ + TTn be its dccompositiou. Since the values of /i((T, tt) are uniquely de- 
termined by the recurrences of Proposition [1] and [2J it is enough to show that Jl 
satisfies the same recurrences. 
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Figure 2. The normal embeddings of cr in tt (see Figure [T]), to- 
gether with their signs. The leaves covered by the embedding are 
represented by black disks, the leaves that are maximal omitted 
are represented by empty circles. Dotted lines represent subtrees 
rooted at a maximal omitted internal node. Note that the leaves 
of such subtrees do not contribute to the sign of the embedding. 

Consider first the case when tti = 1, which is treated by Proposition [T] Let ai + 

h be the decomposition of a, let k — deg(7r) and let £ — deg(cr). This means 

that the leftmost k leaves of T{tt) are all children of the root node, and they form 
a leaf run. Therefore, in any normal embedding, all the k — 1 leaves representing 
772, . . . , TTfe are covered, because they are followers of tti. Necessarily, any element 
of cr that is embedded to one of the first k elements of tt must be one of the first i 
elements of a. Consequently, if fc — 1 > ^, there is no normal embedding of cr into 
TT, and 7l((T, tt) = 0. 

Suppose now that fc — 1 ~ £. Then, in any normal embedding / e N{a,TT), the 
element tti is omitted, the elements representing ai, . . . ,ak^i are embedded on 
7r2,...,7rfc, and the elements of cr>fc_i are embedded to the elements 7r>fe. The 
restriction of / to cr^k-i is a normal embedding /' from the set N{ayk-i, T^yk), and 
conversely, a normal embedding /' from N{ayk-i,'^>k) can be uniquely extended 
into an embedding / € N{a, tt). We then have d{f) = 1 + d(f'), because tti is the 
only maximal omitted leaf of / that is not a maximal omitted leaf of /'. This shows 
that 7l(cr,7r) ^ -7i(cr>fe_i,7r>fe). 

Assume now that k — 1 < £. Let N~^{a,TT) denote the set of normal embed- 
dings of a into tt that cover the element tti , and let A''" (cr, tt) be the set of 
those that omit tti. By the same argument as in the previous paragraph, we 
see that A'^'^(cr, tt) is mapped by a sign-preserving bijection to A^(cr>fc, 7r>fe), and 
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N^{a,T:) is mapped by a sign-reversing bijection to N{ayk-i,'T^>k)- Consequently, 
7l(cr,7r) = 7l(CT>fe,7r>fc) -7i(cr>fe_i,7r>fc). 

These arguments show that fl satisfies the recurrences of Proposition [TJ 

Assume now that tti > 1, which corresponds to the situation of Proposition [21 Let 
TTi + • • • + 7r„ be the decomposition of tt, let txi + • • • + Gm be the decomposition of 
tr, and let k € [n] be the largest integer such that tti = ■ ■ ■ — nk- The n blocks of 
TT correspond precisely to n children of the root of the tree T{Tr), and the leftmost 
k blocks form a non-leaf run. Therefore, each normal embedding / G N(a, tt) must 
cover the leftmost child of the root, which represents tti, but it may omit some of 
its followers, which represent the blocks 7r2, . . . , TTfc. Note that the symbols of a that 
are embedded into tti by / must form a prefix of the form a<i, for some i € [m]. 

For / G N{a,TT), let /(/) G [m] be the largest number i such that all the symbols 
of a<i are embedded into tti, and let J(/) G [fc] be the largest number j such that 
among the leftmost j children of the root of T(7r), only the node representing tti 
is covered. Let Nij be the set {/ G N{a,n): I{f) = i, J{f) = j}. Notice that 
an embedding / G Ni,j decomposes in an obvious way into a normal embedding 
fl G N(cr<i,ni) and a normal embedding /2 G A'^((T>i, 7r>j), and that we have 
d{f) = d{fi) + c?(/2), and hence sgn(/) = sgn(/i)sgn(/2). This decomposition is a 
bijection between Nij and N(cr<i,ni) x N{ayi,TTyj). Consequently, we have the 
identity 

sgn(/) = Y sgn(/i)sgn(/2) =7i(cr<,,7ri)7i(cr>i,7r>j). 

Summing this identity for each i G [m] and each j G [k] , we conclude that 

m k 
i=l j=l 

which is the recurrence of Proposition [21 Therefore, Jticr, tt) — ^{<t, tt). □ 

Let us now state several consequences of Theorem [2H 
Corollary 23. If tt is separable, then /i(l,7r) G {0,1,-1}. 

Proof. The permutation 1 can have at most one normal embedding into tt. Namely, 
if \tt\ > 1, then T(7r) has at least one leaf £ that is not a leader of its leaf run, but 
each of its ancestors is a leader of its non-leaf run. Such a leaf £ must be covered 
by any normal embedding of any permutation into tt. □ 

The next corollary confirms a (more general version of a) conjecture of Steingrimsson 
and Tenner [7 . 

Corollary 24. If ir and a are separable permutations, then |/i((T, 7r)| is at most the 
number of occurrences of a m tt. 

Proof. This follows from the fact that the number of occurrences of cr in tt is clearly 
at least the number of normal embeddings of a into tt. □ 

Using Theorem [22l it is easy to show that for 7r„ = 214365 • • • (2n)(2n — 1), we have 
/i(12,7r„)=n — 1. Thus, the following result. 

Corollary 25. The value of the Mobius function on intervals [cr, tt] is unbounded, 
even for separable permutations a and tt. 



18ALEXANDER BURSTEIN, VfT JELINEK, EVA JELfNKOVA, AND EINAR, STEINGRIMSSON 

Recall that a permutation is layered if it is the concatenation of decreasing se- 
quences, such that the letters in each sequence are smaller than all letters in subse- 
quent sequences. One example is the permutation 21365487, whose layers are shown 
by 21-3-654-87. Sagan and Vatter [5j gave a formula for the Mobius function of 
intervals of layered permutations, and it is easy to see that layered permutations are 
special cases of separable permutations. Namely, a layered permutation is separa- 
ble, and its separating tree has depth 2 (except in the trivial cases of the increasing 
and decreasing permutations), where the children of the root are the layers of the 
permutation, and the grandchildren of the root are all leaves. 

5. Concluding remarks, conjectures and open problems 

We have shown in Corollarv 1191 that /^(cr, tt) can be computed efficiently when a 
and TT are separable. The same argument does in fact apply in a more general 
form: For any hereditary class C of permutations that is a closure of a finite set of 
permutations, there is a polynomial-time algorithm to compute /z((t, tt) for a given 
a and tt in C. We do not know whether such an algorithm also exists for more 
general classes of permutations. 

Bose, Buss and Lubiw [2; have shown that it is NP-hard for given permutations tt 
and a to decide whether tt contains a. In view of this, it seems unlikely that /i((T, tt) 
could be computed efficiently for general permutations a and tt. 

Our results imply that for a separable permutation tt, the Mobius function /i(l,7r) 
has absolute value at most 1. In fact, the class of separable permutations is the 
unique largest hereditary class with this property, since any hereditary class not 
contained in the class of separable permutations must contain 2413 or 3142, and 
/j,(l,2413) = /i(l,3142) = —3. It is natural to consider /i(l,7r) as a function of 
TT, and ask whether this function is bounded on a given class of permutations. By 
Corollary [20l if a hereditary class C is a closure of a finite set of permutations, 
then ^(l,7r) is bounded on C. We do not know if there is another example of a 
permutation class on which this function is bounded. 

On the other hand, we do not have a proof that /i(l,7r) is unbounded on the set 
of all permutations, although numerical evidence suggests that this is the case. 
According to our computations, the sequence of maximum values of l/z(l,7r)| for 

TT S Sn, starting at n — 1, begins 1, —1, 1, —3, 6, —11, 15, —27, —50, —58, 143, 

For these cases (n < 11), there is, up to trivial symmetries, a unique permutation 
for which the Mobius function attains this value. These permutations are 

1, 12, 132, 2413, 24153, 351624, 2461735, 35172846, 
472951836, 46819210357, 361941172105 8 

All of the above permutations are simple (except for 132, but there are no sim- 
ple permutations of length 3). A permutation is simple if it has no segment 
Oifli+i . . . fli+fc where 1 < fc < n — 1 and {0^,0^+1, . . . , ai+k} is a set of consecu- 
tive integers (see [J). Thus, in some (imprecise) sense, simple permutations are 
the opposite of (skew) decomposable permutations (and, in particular, separable 
permutations). In particular, a simple permutation can neither be decomposed nor 
skew decomposed. We are not able to compute /i(l,7r) for all permutations tt of 
length 12, but for simple permutations vr the maximum value of /i(l,7r) is —261, 
for vr = 4 7 2 10 5 1 12 8 3 11 6 9. 

In light of Corollary [51 to show that |^(l,7r)| is unbounded, it would suffice to 
show that the maximum of |/z(l,7r)| for permutations tt of length n, for any n, 
is attained only by a permutation tt that does not start with 1. In that case 
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1^(1,1 + 7r)| = 1/^(1, 7r)|, SO there would be a permutation r of length n + I for 

which |/i(l,T)| > 1^(1,1+^)1 = \nil,7T)\. 

Question 26. For which permutation classes C is the function /i(l,7r) bounded on 
C? Is /i(l,7r) unbounded on the set of all permutations? Can non-trivial upper or 
lower bounds 6e /ottnii /or maxTre^,, |/i(l, 7r)|, as a function of n? 

We have exhibited several classes of intervals whose Mobius function is zero (and 
more were presented in [7]). Can the following question be answered precisely? 

Question 27. When is ^(cr, n) = 0? 

For separable permutation tt, we have shown that |/z((t, 7r)| is at most cr(7r), that 
is, the number of occurrences of a in tt. This is not true for non-separable tt, even 
when (T = 1, as shown above. However, it might be possible to bound |/i((t, 7r)| as 
a function of cr(7r). 

Question 28. Is there an upper bound for |/i(cr, 7r)| that only depends on (j{-k)? 
The following conjecture has been verified for n < 10. 

Conjecture 29. The maximum value of the Mobius function /i(cr, tt) for separable 
permutations a and tt, where tt has length n > 3, is given by 



This maximum is attained by the permutation tt that starts with its odd letters in 
increasing order, followed by the even letters in decreasing order, and a of the same 
form and length 2 ■ [(n + 1/2) J if the length of n is 2n, and 2 • [(n + 1/2)J — 1 if 
the length of n is 2n — 1. 

As an example, m(13542, 135798642) = 15 = C^l^'^)- 

Finally, we mention some questions about the topology of the order complexes 
of intervals in the poset V. (For definitions, see [B]). Given an interval [cr, tt], let 
A((T, tt) be the order complex of the poset obtained from [a, tt] by removing a and n. 

Question 30. (1) For which a and tt does A((T, tt) have the homotopy type of 
a wedge of spheres ? 

(2) Let F be the subcomplex of A(a,T:) induced by those elements r of [cr, tt] for 
which ii{a,T) — 0. Is T a pure complex? 

(3) If a occurs precisely once in n, and ^{a,n) = ±1, is A(cr, tt) homotopy 
equivalent to a sphere? 

(4) For which a and tt is A(ct, tt) shellable? 

We should mention that for a = 231 and tt = 231564, the order complex A(f7, tt) is 
not shellable; it consists of two connected components, each of which is contractible. 
However, removing from [231, 231564] all elements t with /x(231, r) = 0, we obtain 
a shellable complex, namely a four-element boolean algebra. For parts ^ and ([3]) 
in Question 1301 we know no counterexamples. Since we have so far only examined 
intervals of low rank, our evidence is not strong. 
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